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Abstract. Here we investigate specifically the transient of a synchronizing system, 
considering synchronization as a relaxation phenomenon. The stepwise establishment 
of synchronization is studied in the system of dynamically coupled maps introduced by 
Ito & Kaneko (2001 Phys. Rev. Lett. 88 028701, 2003 Phys. Rev. E 67 046226), where 
the plasticity of dynamical couplings might be relevant in the context of neuroscience. 
We show the occurrence of logarithmically slow dynamics in the transient of a fully 
deterministic dynamical system. 



PACS numbers: 05.45.Xt, 89.75.-k 
1. Introduction 

The importance of transients in the dynamics of complex systems is manifold : for 
example, the transient state can be more relevant than the equilibrium state if most or 
more of the time is spent in the former. Also, relaxation dynamics can inform on the 
underlying energy, fitness, or cost landscape of a system (Wales 2004), and thus help to 
understand it better whole. 

In this paper we are interested in studying synchronization, a dynamical property 
of networks that is widely observed in fields such as optics, chemistry, biology and 
ecology, for instance in the brain (Gray et al. 1989, Varela et al. 2001) or in fireflies. 
Synchronization has been analysed for many physical systems (Pecora et al. 1997, 
Pikovsky, Rosenblum & Kurths 2001). The onset of synchronization is of general 
relevance. Similarity with relaxation, for example in spin glasses, or in superconductors, 
can be used to study synchronization. Transients can be used to probe the properties 
of synchronizing systems (Arenas et al. 2006). However, there has been little research 
so far on synchronization as a relaxation phenomenon (Abramson 2000, Manrubia &; 
Mikhailov 2000). The aim of this paper is to systematically study the transient of a 
synchronizing system. 

A specific form of relaxation dynamics are glassy dynamics, when the transient is 
extremely long (Jensen & Sibani 2007). Furthermore, in a number of systems with glassy 
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dynamics the special case of logarithmically slow dynamics has been observed. However, 
so far all of these systems were stochastically driven. Here we show the occurrence of 
logarithmically slow dynamics in the transient of a fully deterministic dynamical system. 
In the first part of the paper we explain the model, a globally coupled map (GCM), 
with adaptive coupling which is inspired by the plasticity of synapses. We then study 
characteristics of its transients for a range of parameters. Finally we show that for some 
parameters the transient is logarithmically slow and that it can be explained by a simple 
model. 



2. The Ito-Kaneko model of synchronization 

The Ito-Kaneko model (Ito & Kaneko 2001, Ito & Kaneko 2003) is a globally coupled 
map (GCM), a coupled simultaneous system of N logistic equations. The individual 
maps or units x % are defined and coupled as follows : 



N 

x\ +1 =(l-c)f(xt)+cJ2^f(4 

3=1 

f(x) = ax (1 — x) 
w t+l 



(1) 



£;Li[i + ^(^)K 

g(x,y) = l- 2\x-y\ , 

where a is the logistic equation parameter and c is the coupling parameter. The 
coupling is further tuned by weights w % \ which are dynamical variables as well. The 
function g, scaled by a parameter S, defines a Hebbian update of the connection weights, 
by reinforcing the connections between similar units. This Hebbian plasticity of the 
couplings is inspired by the synaptic plasticity which enables nervous systems to learn 
(Antonov et al. 2003). 

The system exhibits three different long-term behaviours which can be classified 
into three phases, depending on the parameters. In the coherent phase C, all units 
synchronize, forming one synchronized cluster containing all units. In the ordered phase 
O, the set of units is partitioned into subsets or clusters Ck within which there is 
synchronization or which contain single units not synchronized with any other unit. 
In the following, in line with (Ito & Kaneko 2001, Ito & Kaneko 2003, Manrubia & 
Mikhailov 2000), we will call the number of parts in this partition the cluster number. 
Finally, in the disordered phase D, no synchronization at all is achieved, forming a 



partition with a cluster number of value N. The phase diagram in figure 1(a) shows the 
boundaries between predominant phases in a part of the parameter space. 
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Figure 1. (a) Part of the phase diagram of the Ito-Kaneko model (b) Length of 
transient (c) Time t w until the variation of weights falls below 0.1% (all averaged 
over 25 realizations). 



3. Synchronization transients 

It has been found previously that in a static links version of the Ito-Kaneko model 
its transient is exponential in the ordered regime O and that its length diverges 
when the border O/D between the ordered and disordered regimes is approached 
(Abramson 2000, Manrubia & Mikhailov 2000). Stretched exponential decay of 
correlation functions has been found in a similar system (Katzav & Cugliandolo 2005). 

In the following we study the transient in detail, and focus on the way events during 
the transient are simulated and detected. The system (CQ) was simulated numerically, 
each initial x % being randomly chosen in the interval [0,1]. The initial are set to 
l/(iV — 1). A higher precision simulation method similar to Pikovsky et al. (Pikovsky, 
Popovych & Maistrenko 2001) was used in order to avoid synchronization artifacts due 
to limited numerical precision. Two units x % and x J were considered to be synchronized 
if \x l — x->\ < 10~ 270 . Due to the resulting increased computation times we focused on 
a part of the parameter space, shown in figure [TJ During each run of a simulation, 
the time steps tk at which two units synchronized were recorded. We term these tk 
synchronization events. As the successive synchronization is analyzed in analogy to the 
investigation of avalanches and quakes in (Anderson et al. 2004), where a set of events is 
considered as one quake, here any synchronization events less than At = 10 time steps 
apart are considered together as a single synchronization event. 

In figure 11(a) we show as an example the time evolution of the number of 
synchronized clusters for a single simulation run. The number of clusters drops stepwise 
from iV = 100 during a long transient. We note N SJQC (t) the number of synchronization 
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Figure 2. (Top) Divergence of transient length as the border to the disordered 
region is approached, with a = 3.9. Shown are the time of the first synchronization 
event t\ (solid black), of the 25th synchronization event £25 (dark gray), and of the 
last synchronization event too (light gray), and t w (dotted black when w converges 
in all realizations, dash-dotted otherwise). (Bottom) Corresponding number of 
synchronization clusters (averaged over 25 simulation runs). 

events until time step t. If, after a synchronization event at time step t^, no further 
synchronization events happened for a duration of 4tfc, the simulation was stopped. We 
define the synchronization transient as the time from the beginning of the simulation 
until the last synchronization event. 

The dependence of the transient on the parameters a and c is shown in figure [H(b) 
and (c). The main feature is that the length of the transient, and also the time until 
the connection weights w 13 stabilize, increases as the the overall coupling c decreases. 
Figure [2] shows again in detail how the length of transient diverges as c diminishes, with 
the parameter of the logistic function a = 3.9 constant. This makes intuitively sense 
as the system changes from a regime with some synchronization (ordered) to a regime 
with no synchronization at all (disordered). For the rest of this paper, a is fixed to 3.9 . 

A theoretical value for the coupling co/d at which the length of the transient 
diverges can be analytically derived for a system in which the weights w l] stabilize, see 
(Ito & Kaneko 2001, Ito & Kaneko 2003). The divergence of the transient corresponds 
to the border between the ordered and disordered regimes and can thus be calculated 
by studying the stability of the ordered state, using the transversal Lyapunov exponent. 
At the considered border the system only forms synchronization clusters of size up to 
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(a) (b) 

Figure 3. Time dependence of -/V sync (i oc ) 
(solid) with variance (dashed) and fitted 
form cxp(-t - 28 ) (dotted), c = 0.28, 0.22, 



(c) 

— N sync (t) averaged over 25 simulation runs 
exponential or stretched exponential of the 
0.158 (from left to right). 



2, i.e. only isolated pairs of synchronized units are formed. The transversal Lyapunov 
exponent for a system in which the weights stabilize is then (Ito & Kaneko 2001, Ito & 
Kaneko 2003): 

A± = In |1 -2c\ + A/, (2) 

where Af is the Lyapunov exponent of f(x), the logistic function. For a = 3.9, the 
Lyapunov exponent of the logistic function calculated by simulation is Af « 0.73 and we 
obtain a theoretical value c ~ 0.26 . In our system the weights do not stabilize, which 
seems to enable synchronization at lower c values, as the border lies at co/d ~ 0.15 . 

We studied the transient in a logarithmic timescale for c values close to and 
far from Co/d- In figure [3] we plotted N sym .(t oa ) — N sync (t) against time in order to 
detect exponential transients of the form A^ sync (t 00 ) — N sync (t) oc e _/3t , as described in 



(Abramson 2000, Manrubia & Mikhailov 2000). For c = 0.28, shown in figure 3(a^ 



the transient is indeed exponential. However, as c decreases, the transient turns more 



irregular, see figure 3(b) (c = 0.22). Eventually, for c = 0.158, very close to cq/d, 



the transient is extremely long and settles into the stretched exponential shape shown 
in figure 3(c) Stretched exponential relaxation has been observed in related systems 
(Katzav & Cugliandolo 2005). Also the evolution of the variance of the considered 
quantity is different depending on whether c away and very close to the border. The 
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difference in the shape of the transient and its variance indicates that a different process 
is underlying the system at the border O/D. In the following we study further the 
transient statistics at the border O/D. 



4. Logarithmically slow transients 

We are especially interested in the unusual dynamics at the border O/D. Also, this 
border separates ordered from disordered behaviour and is interesting because of the 
relevance of computation at the edge of chaos in neural nets (Natschlager et al. 2005). 
The transient at the border is characterized by its extreme length. 

Extremely long transients have also been observed in other many component 
systems with glassy dynamics (Jensen & Sibani 2007). There, the time span needed 
to reach a steady, time independent state is often far beyond experimentally accessible 
time scales. For example, when melted alloys are cooled down they typically retain the 
amorphous arrangement characteristic of the liquid high temperature phase while the 
molecular mobility decreases many orders of magnitude, rendering it near impossible to 
reach thermodynamic equilibrium. However, over short time scales the system properties 
may appear to be time independent as in thermal equilibrium. Only when several orders 
of magnitude of time scales are covered the slow change of macroscopic characteristic 
properties with time can be resolved directly. 

A superposition of 25 simulations runs for c = 0.158 close to co/d covering the 
whole very long transient up to t = 10 7 is shown in figure HI Due to computational 
complexity no value closer to co/d was used. Interestingly, the transient is linear on a 
logarithmic timescale for intermediate times. In the following we describe an approach 
to modeling the transient. 

A simple model would be to assume that the rate of synchronization events 
is proportional to the rate at which any two units have close values by chance, 
i.e. 'collide'. However, at low overall coupling c, units x l can only synchronize with 
a single additional unit and form pairs. Thus units that are already paired up are not 
available anymore to synchronize with other units. So simplifying further, if we note 
the number of synchronization events N sync (t), and the number of unsynchronized units 
is approximately iV — N sync (t), we obtain: 

= k(N - N sync (t)) 2 (3) 

and 

N sync(t) = N-j-^- t . (4) 

We further assume that only an effectual subset of the N units can actually synchronize 
due to the low overall coupling c, so we replace iV by an effectual, i.e. actually operating 
number of units N e g in the above formula. The predicted N sync (t) agrees well with the 
simulated data for N cS = 24.5 and k = 6 ■ 10~ 7 , see figure @|b). However, it is not 
possible to find a reasonable fit of the simulated N sync (t) to (TjJ for higher values of c. 
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Figure 4. Synchronization transient near the border O/D, with c = 0.158 . (a-insert) 
Time dependence of the number of synchronization clusters for a single simulation run. 
(b) Time dependence of the number of synchronization events for 25 simulation runs 
(thin lines) and corresponding variance (dashed), a linear fit and a fit of the simple 
model described by ^ (thick lines). 

We also observed that N sync (t) is linear on a logarithmic timescale for intermediate 
times. This might be related to the log-time dependence that has been previously 
obtained for non-homogeneous Poisson processes in logarithmic time (log-Poisson). The 
hypothesized mechanism behind these processes is inspired by intermittency studies 
of fluctuations in glassy systems, that have demonstrated that large intermittent 
fluctuations are responsible for the deviations from equilibrium statistics (Buisson 
et al. 2003). It was suggested that abrupt and irreversible moves from one metastable 
configuration to another, so-called quakes, are a result of record sized fluctuations. The 
assumption that the metastable attractors typically selected by the glassy dynamics 
have marginally increasing stability means that a fluctuation bigger than any previously 
occurred fluctuation, i.e. a record-sized fluctuation, can induce a quake (Sibani & 
Dall 2003, Sibani & Littlewood 1993). Quakes lead to entrenchment into gradually 
more stable configurations, and carry the average drift of the dynamics. The quakes 
have a similar effect on a logarithmic time scale, which might be modeled by a Poisson 
process in logarithmic time. 

Log-Poisson processes have been observed in the NK model of evolution (Sibani 



Logarithmically slow onset of synchronization 



8 



& Pedersen 1999), charge- density waves (Sibani & Littlewood 1993), and further in 
spin glasses, supercooled magnet relaxation, and the Tangled Nature evolution model 
(Anderson et al. 2004). While in a Poisson process the probabilities for events tk are 
characterized by (van Kampen 2007): 

e~ XT (\r) k 

P[N(t + r)-N(t) = k] = e ^ } (5) 

and 

P[t k - 4_i < x] = 1 - e~ Xx , (6) 
in analogy, in a log- Poisson process we have (Sibani & Dall 2003): 

P[N(t + r) - N(t) = k) = i (*±l) " (a^±I) * (7) 

P[ln(t fc /4_ 1 )<^] = l-e- Aa; . (8) 

In both cases, it is easy to show that the PDF of the number of events N(t) and 
its variance are equal. Thus in figure [3] A^ sync (t 00 ) — N sync (t) and its variance should 
be symmetric, which is observed indeed only up to some error on the border O/D 



(figure 3(c)) while away from the border (figure [3](a) and (b)) iVsync(£cio) — N sync (t) and 
its variance seem unrelated. 

Thus the process described by (jlj) seems to be an alternative way to obtain log-time 
dependence in an intermediate time regime. We can further corroborate this finding by 



studying the cumulative distribution of (t k — see figure 5(b) For a static 

Poisson process, this distribution would be a step function. Instead, it agrees well 
with the theoretic distribution for a log-Poisson process, P((tk — < x) — 

1 — (1 + x)~ x with A = 3.5. The support for the distribution at low {t k —tk-i)/tk-i 
comes from the initial phase of the transient, where the initial conditions dominate, 
which might explain the discrepancy between the theoretical and simulated distribution 



there. Also, as shown in figure 5(a) , the value of (t k — £fc-i)/£&-i stays within a relatively 
small range over a range of orders of magnitude of tk, while it would quickly drop to zero 
in a static Poisson process. In figure 5(c) we show the cumulative distribution of tk/tk-v 



This distribution also agrees well with the theoretical cumulative distribution (jHJ) for a 
log-Poisson process with A = 3.5 and seems to corroborate that the synchronization 
process in this model is an alternative way to obtain logarithmically slow relaxation 
dynamics. 

System size can be relevant to the shape of the transient of a system. A 
superposition of the transients for different system sizes, normalized by the final number 
of synchronization events, and for c = 0.158 close to co/d and c = 0.18, is shown 
in figure [6](a) and (b). Due to computational complexity, the sampling size is small, 
especially for iV = 200 . The simulations show that the system size dependence close to 
Co jB seems to be a shifting of the transient to faster synchronization. The shifting is 
close to what (J3J suggests. Also, for c = 0.18, further away from cq/d, the transients 
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Figure 5. (a-insert) Distribution, (b) cumulative distribution of (tk — £fc-i)Afc-i 
for c = 0.158 (solid), and theoretic distribution for a log-Poisson process (dotted) (c) 
Cumulative distribution of tfc/tfc_i close to the border O/D with c = 0.2 (light gray), 
0.16 (dark gray) and 0.158 (black), and fitted theoretic distribution for a log-Poisson 
process (dashed). 
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Figure 6. System size dependence of the transient, (a) Transients for c = 0.158 and 
iV = 25, 50, 100, and 200 (solid gray to black) and corresponding theoretical transients 
using (gl), N eS = 7,14,24.5, and 61 and k = 6 • 10~ 7 (dotted), (b) Transients for 
c = 0.18 and N = 25, 50, and 100 (solid) and variance (dashed). 
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are almost equivalent for different system sizes when normalized by the final number 
of synchronization events. Thus also here the system synchronizes faster for bigger 
system sizes. The faster synchronization for increasing system size can be rationalized 
using the same idea that led to (jl]), as follows: since the values of the units x 1 are 
constrained to the interval [0, 1] of the logistic map, increasing the system size increases 
the number of units in [0, 1] and therefore increases the number of collisions, i.e. the 
speed of synchronization. 

5. Conclusion 

In conclusion, we have studied synchronization as a relaxation phenomenon. The 
transient of synchronization in a coupled map model was found to drastically depend 
on the amount of overall coupling. For an overall coupling at the border to disorder, the 
transient was found to be logarithmically slow at intermediate times. This behaviour 
has been found in other systems exhibiting record statistics, where the dynamics is 
termed log-Poisson and arises because record fluctuations become increasingly rare. In 
the model of the present system the dynamics may arise by an alternative way : it 
can be simply explained by (for intermediate times) it getting more and more difficult 
to find a new synchronization partner and actually synchronize as the system ages. 
Interestingly, this kind of dynamics has been found otherwise in noise-driven systems. 
The coupled map model used in this paper has no noise term, however, it uses the 
logistic map, which has been studied as a noise generator since Ulam & Neumann 
(Ulam & von Neumann 1947). Thus the use of the logistic map might be an ingredient 
for the appearance of logarithmically slow dynamics. In a similar system it was shown 
that a coupled map lattice is equivalent to a system of stochastic PDE (Katzav & 
Cugliandolo 2005). 

One example for the relevance of the synchronization observed in GCM is neural 
networks. The speed of synchronization in neural networks is important (Kopell & 
Ermentrout 2004), as it presumably is related to the response time of the brain. Namely, 
if we assume, along with Varela et al. (Varela et al. 2001), that the state of the brain is 
given by its state of synchronization, then the time to swap from one synchronization 
pattern to another seems to determine how fast the brain can react. Also, the behaviour 
at the edge between partial synchronization and disorder is especially interesting because 
of the relevance of computation at the edge of chaos (Natschlager et al. 2005). 
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